The present invention relates to filter operation apparatus which are used in image coding methods.
As the international standard systems as to image coding, there are MPEG (Moving Picture Experts Group) according to ISO (International Standardization Organization), ITU-T (International Telecommunication Union-Telecommunication Sector) recommendation H.261 and the like. Although the use of MPEG is increasing in recent years, the conventional H.261 is still utilized as always in the present circumstances. That is, both of the systems exist together at present. Therefore, filter operation apparatus which can be used in both of the systems are required.
Initially, these systems are briefly described.
In the coding method according to MPEG of ISO, DCT (Discrete Cosine Transform) which performs orthogonal transform as a compression method utilizing a spatial correlation is used for bidirectional motion compensation inter-frame prediction as a method utilizing a temporal correlation. Half-pixel motion compensation used in this method is a simple method of averaging two pixels when the position of a pixel to be predicted is between the two pixels and averaging four pixels when the position is among the four pixels. Accordingly, the half-pixel motion compensation not only increases the prediction accuracy but also has a function of a spatial low-pass filter. Further, when the four pixels are averaged, there are some cases where an average pixel value in the horizontal direction is obtained and then an average pixel value in the vertical direction with respect to the obtained average pixel value is obtained as an average of the four pixel values.
In the coding method according to ITU-T recommendation H.261, compression methods utilizing the spatial correlation or temporal correlation are used as in MPEG system. However, in this coding method, an intra-loop filter, i.e., a spatial low-pass filter is used to avoid the situation that the distortion occurring due to the quantization is stored in a prediction memory, whereby the degradation in image quality is increased and the prediction efficiency is decreased.
In the processing of this intra-loop filter, the weighting is performed according to the positions of pixels, as shown in FIG. 21.
To be specific, a pixel value P of a pixel p in each region shown in FIG. 21 obtains a new pixel value Pxe2x80x2 in a following manner.
As for a pixel value P of a pixel p in a region 2100,
Pxe2x80x2=16xc3x97(P/16)xe2x80x83xe2x80x83(1)
As for a pixel value P of a pixel p in a region 2101,
Pxe2x80x2=((4xc3x97A)+(8xc3x97P)+(4xc3x97B))/16xe2x80x83xe2x80x83(2)
As for a pixel P in a region 2102,
Pxe2x80x2=(A+(2xc3x97B)+C+(2xc3x97D)+(4xc3x97P)+(2xc3x97E)+F+(2xc3x97G)+H))/16xe2x80x83xe2x80x83(3)
In the expressions (1)xcx9c(3), axcx9ch each denote a pixel adjacent to each of the pixels p shown in FIG. 21 and Axcx9cH each denote a pixel value of each of the pixels axcx9ch shown in FIG. 21. The processing for the weighting is performed twice, i.e., in the horizontal direction and vertical direction.
Usually, the conventional filter operation apparatus which realizes the filter processing of the above-mentioned coding method according to MPEG of ISO and coding method according to ITU-T recommendation H.261 in an apparatus is constructed so that a part of the apparatus is shared in the filter processing of these two coding methods, thereby avoiding the increase in the scale of hardware. If each of parts associated with these two coding methods is independently provided in one apparatus, the scale of the hardware is adversely increased.
The structure and operation of the conventional filter operation apparatus X are briefly described with reference to drawings.
FIG. 14 is a diagram simply illustrating a structure of the conventional filter operation apparatus X having a horizontal half-pixel motion compensation and horizontal. intra-loop filtering means 1400 and a vertical half-pixel motion compensation and vertical intra-loop filtering means 1401. FIG. 15 is a block diagram illustrating a structure of the horizontal half-pixel motion compensation and horizontal intra-loop filtering means 1400. FIG. 16 is a block diagram illustrating a structure of the vertical half-pixel motion compensation and vertical intra-loop filtering means 1401.
Initially, the structure of the horizontal half-pixel motion compensation and horizontal intra-loop filtering means 1400 is described with reference to FIG. 15. The horizontal half-pixel motion compensation and horizontal intra-loop filtering means 1400 comprises a first pixel delay means 1500 for delaying an input pixel D21 for a predetermined period and outputting a delayed pixel, a multiplication means 1501 for multiplying the input pixel D21 by two and outputting a multiplied pixel, a first selection means 1502 for selectively outputting one of the input pixel D21 and the pixel which is obtained by multiplying the input pixel D21 by two by the multiplication means 1501 in accordance with a mode switch signal S21 for switching the mode between xe2x80x9chalf-pixel motion compensation modexe2x80x9d and xe2x80x9cintra-loop filtering modexe2x80x9d, a second selection means 1503 for selectively outputting one of xe2x80x9c0xe2x80x9d and the output of the first pixel delay means 1500 in accordance with a first intra-loop filter control signal S22, a third selection means 1504 for selectively outputting one of the output of the first selection means 1502 and the output of the second selection means 1503 in accordance with a half-pixel motion compensation control signal S23, a first addition means 1505 for adding the output of the second selection means 1503 and the output of the third selection means 1504 and outputting output data D22, a second pixel delay means 1506 for delaying the output signal of the first addition means 1505 for a predetermined period and then outputting output data D23, a fourth selection means 1507 for selectively outputting one of the input pixel D21 and the output signal of the second pixel delay means 1506 in accordance with a second intra-loop filter control signal S24, a second addition means 1508 for adding the output of the second pixel delay means 1506 and the output of the fourth selection means 1507 and outputting output data D24, and a third pixel delay means 1509 for delaying the output signal of the second addition means 1508 for a predetermined period and then outputting output data D25.
Next, the operation of the horizontal half-pixel motion compensation and horizontal intra-loop filtering means 1400 is described with respect to the cases of half-pixel motion compensation and intra-loop filtering, respectively. Assume that the first pixel delay means 1500, the second pixel delay means 1506, and the third pixel delay means 1509 each delay the data for one clock.
A format of input data in the case of half-pixel motion compensation is described with reference to FIG. 3. As shown in FIG. 3(a), the format of input data which are subjected to the horizontal processing for generating 8xc3x978 half-pixels in the horizontal processing consists of 9xc3x979 pixels. As shown by arrows in FIG. 3(a), the processing is performed successively from left to right and from top to bottom on a two-dimensional space.
To be more specific, the processing is performed in the order of Axe2x86x92Bxe2x86x92Cxe2x86x92 . . . xe2x86x92Ixe2x86x92Jxe2x86x92Kxe2x86x92 . . . xe2x86x92Z as shown in FIG. 3(a). Timing charts of FIGS. 17 show this operation in more detail. In FIGS. 17, S21 denotes a control signal for controlling the first selection means 1502 shown in FIG. 15 to output xe2x80x9caxe2x80x9d. S22 denotes a control signal for controlling the second selection means 1503 shown in FIG. 15 to output xe2x80x9cbxe2x80x9d. S23 denotes a control signal for controlling the third selection means 1504 shown in FIG. 15 to output xe2x80x9cbxe2x80x9d. S24 denotes a control signal for controlling the fourth selection means 1507 shown in FIG. 15 to output xe2x80x9caxe2x80x9d.
A format of input data in the case of intra-loop filtering is described with reference to FIG. 4. As shown in FIG. 4(a), the format of input data which are subjected to the horizontal processing of intra-loop filtering consists of 8xc3x978 pixels. As shown by arrows in FIG. 4(a), the processing is performed successively from left to right and from top to bottom on a two-dimensional space.
To be more specific, the processing is performed in the order of Axe2x86x92Bxe2x86x92Cxe2x86x92 . . . xe2x86x92Fxe2x86x92Gxe2x86x92Hxe2x86x92Ixe2x86x92Jxe2x86x92 . . . xe2x86x92Z as shown in FIG. 4(a). Timing charts in FIGS. 18 show this operation in more detail. In FIGS. 18, S21 denotes a control signal for controlling the first selection means 1502 shown in FIG. 15 to output xe2x80x9cbxe2x80x9d. S23 denotes a control signal for controlling the second selection means 1503 shown in FIG. 15 to output xe2x80x9caxe2x80x9d. S22 denotes a control signal for controlling the third selection means 1504 shown in FIG. 15 to output xe2x80x9cbxe2x80x9d during the period of times t0xcx9ct1, output xe2x80x9caxe2x80x9d during the period of t1xcx9ct7, output xe2x80x9cbxe2x80x9d during the period of t7xcx9ct9, output xe2x80x9caxe2x80x9d during the period of t9xcx9ct10, and thereafter repeat the above-mentioned operations of t0xcx9ct10. S24 denotes a control signal for controlling the fourth selection means 1507 shown in FIG. 15 to output xe2x80x9caxe2x80x9d during the period of times t1xcx9ct2, output xe2x80x9cbxe2x80x9d during the period of times t2xcx9ct8, output xe2x80x9caxe2x80x9d during the period of t8xcx9ct10, and thereafter repeat the above-mentioned operations of t1xcx9ct10.
The structure of the vertical half-pixel motion compensation and vertical intra-loop filtering means 1401 is described with reference to FIG. 16. The vertical half-pixel motion compensation and vertical intra-loop filtering means 1401 comprises a first pixel delay means 1600 for delaying an input pixel D31 for a predetermined period (one clock) and outputting output data D32, a first delay means 1601 for delaying the input pixel D32 by 8 pixels (corresponding to 8 clocks) and outputting output data D33, a multiplication means 1602 for multiplying the input pixel D31 by two and outputting the multiplied pixel, a first selection means 1603 for selectively outputting one of the input pixel D31 and the pixel which is obtained by multiplying the input pixel D31 by two with the multiplication means 1602 in accordance with a mode switch signal S31 for switching the mode between xe2x80x9chalf-pixel motion compensation modexe2x80x9d and xe2x80x9cintra-loop filtering modexe2x80x9d, a second selection means 1604 for selectively outputting one of xe2x80x9c0xe2x80x9d and the output data D33 in accordance with a first intra-loop filter control signal S32, a third selection means 1605 for selectively outputting one of the output of the first selection means 1603 and the output of the second selection means 1604 in accordance with the half-pixel processing control signal S33, a first addition means 1606 for adding the output of the second selection means 1604 and the output of the third selection means 1605 and outputting output data D34, a second pixel delay means 1607 for delaying the output signal of the first addition means 1606 for a predetermined period (corresponding to one clock) and outputting output data D35, a second delay means 1608 for delaying the output data D35 by 8 pixels (corresponding to 8 clocks) and outputting output data D36, a fourth selection means 1609 for selectively outputting one of the output data D35 and output data D36 in accordance with the above-described mode switch signal S31, a fifth selection means 1610 for selectively outputting one of the output data D32 and the output data of the fourth selection means 1609 in accordance with a second intra-loop filter control signal S34, a second addition means 1611 for adding the output data of the fourth selection means 1609 and the output data of the fifth selection means 1610 and outputting output data D37, a third pixel delay means 1612 for delaying the output data of the second addition means 1611 for a predetermined period (corresponding to one clock) and outputting output data D38, and a division means 1613 for dividing the output data D38 by 16 and outputting the divided data.
Then, the operation of the vertical half-pixel motion compensation and vertical intra-loop filtering means 1401 is described with respect to the cases of half-pixel motion compensation and intra-loop filtering, respectively.
A format of input data in the case of half-pixel motion compensation is described with reference to FIG. 3. Like in the case of the horizontal processing, the input data format consists of 9xc3x979 pixels as shown in FIG. 3(a). As shown by arrows in FIG. 3(a), the processing is performed successively from left to right and from top to bottom on a two-dimensional space.
To be more specific, the processing is performed in the order of Axe2x86x92Bxe2x86x92Cxe2x86x92 . . . xe2x86x92Ixe2x86x92Jxe2x86x92Kxe2x86x92 . . . xe2x86x92Z as shown in FIG. 3(a). Timing charts in FIGS. 19 show this operation in more detail. In FIGS. 19, S31 denotes a control signal for controlling the first selection means 1603 and the fourth selection means 1609 shown in FIG. 16 to output xe2x80x9caxe2x80x9d. S32 denotes a control signal for controlling the-second selection means 1604 shown in FIG. 16 to output xe2x80x9cbxe2x80x9d. S33 denotes a control signal for controlling the third selection means 1605 shown in FIG. 16 to output xe2x80x9cbxe2x80x9d. S34 denotes a control signal for controlling the fifth selection means 1610 shown in FIG. 16 to output xe2x80x9caxe2x80x9d.
A format of input data in the case of intra-loop filtering is described with reference to FIG. 4. Like the input data format in the case of the horizontal processing of the intra-loop filtering, the format of the input data which are subjected to the vertical processing consists of 8xc3x978 pixels as shown in FIG. 4(a). As shown by arrows in FIG. 4(a), the processing is performed successively from left to right and from top to bottom on a two-dimensional space.
To be more specific, the processing is performed in the order of Axe2x86x92Bxe2x86x92Cxe2x86x92 . . . xe2x86x92Fxe2x86x92Gxe2x86x92Hxe2x86x92Ixe2x86x92Jxe2x86x92 . . . xe2x86x92Z as shown by the arrows in FIG. 4(a). Timing charts in FIGS. 20 show this operation in more detail. In FIGS. 20, S31 denotes a control signal for controlling the first selection means 1603 and the fourth selection means 1609 shown in FIG. 16 to output xe2x80x9cbxe2x80x9d. S32 denotes a control signal for controlling the second selection means 1604 shown in FIG. 16 to output xe2x80x9caxe2x80x9d during the period of times t0xcx9ct8, output xe2x80x9cbxe2x80x9d during the period of t8xcx9ct56, and output xe2x80x9caxe2x80x9d during the period of t56xcx9ct64. S33 denotes a control signal for controlling the third selection means 1605 shown in FIG. 16 to output xe2x80x9caxe2x80x9d. S34 denotes a control signal for controlling the fifth selection means 1610 shown in FIG. 16 to output xe2x80x9caxe2x80x9d during the period of times t9xcx9ct17, output xe2x80x9cbxe2x80x9d during the period of t17xcx9ct65, and output xe2x80x9caxe2x80x9d during the period of t65xcx9ct73. (Here, the time subsequent to t22 is not shown in FIG. 20.)
As described above, in the conventional filter operation apparatus realizing the half-pixel motion compensation and the intra-loop filtering, different units are required respectively for the horizontal processing and the vertical processing, whereby the scale of hardware is increased.
The present invention is made in view of the above-described problems. In the present invention, attentions are paid to the fact that a horizontal processing apparatus and a vertical processing apparatus in the processing of half-pixel motion compensation and intra-loop filter for input pixel data can share a core operation unit. When a part which is in the operation unit and interferes the sharing, for example a part for adjusting the transition timing of data is replaced with a unit which adjusts the transition timing of data by switching the mode in a preprocessor, the operation unit in the horizontal processing unit and the vertical processing unit can be shared. Therefore, the filter operation apparatus which can decrease the scale of hardware can be provided.
A filter operation apparatus of first embodiment of the present invention processes input pixel data according to one of a first filter processing and a second filter processing which is different from the first filter processing, and comprises at least first pixel delay means for delaying the input pixel data for a predetermined period and outputting the delayed data; second pixel delay means for delaying the output data of the first pixel delay means for a predetermined period and outputting the delayed data; first multiplication means for multiplying the output data of the first pixel delay means by four and outputting the multiplied data; filter processing switch signal generation means for generating a filter processing switch signal for performing switching to process the input pixel data according to one of the first filter processing and the second filter processing; second multiplication means for multiplying the output data of the first pixel delay means by one when the input pixel data are processed according to the first filter processing in accordance with the filter processing switch signal, and multiplying the output data of the first pixel delay means by two when the input pixel data are processed according to the second filter processing in accordance with the filter processing switch signal; first selection means for selectively outputting one of xe2x80x9c0xe2x80x9d and the output data of the second pixel delay means in accordance with the filter processing switch signal; addition means for adding the input pixel data, the output data of the second multiplication means, and the output data of the first selection means; selection control signal generation means for outputting a selection control signal in accordance with the filter processing switch signal; second selection means for selectively outputting one of the output data of the first multiplication means and the output data of the addition means in accordance with the selection control signal; mode switch signal output means for outputting a mode switch signal for switching a mode between a horizontal processing mode in which the input pixel data are processed in a horizontal direction and a vertical processing mode in which the input pixel data are processed in a vertical direction; multiplier factor control signal generation means for outputting a multiplier factor control signal on the basis of the filter processing switch signal and the mode switch signal; third pixel delay means for delaying the output data of the second selection means for a predetermined period and outputting the delayed data; and third multiplication means for multiplying the output data of the third pixel delay means by one of one, xc2xd, and {fraction (1/16)} in accordance with the multiplier factor control signal which is output by the multiplier factor control signal generation means.
This filter operation apparatus can share the operation unit in the horizontal processing apparatus and the vertical processing apparatus for the input image data. Therefore, the circuit scale of the hardware part can be reduced, whereby the scale of the filter operation apparatus can be reduced.
According to second embodiment, in the filter operation apparatus of first embodiment, it is a preferable embodiment that the addition means add an output of a register which contains values for performing xe2x80x9croundingxe2x80x9d, to one of an operation result of the first filter processing, an operation result of the second filter processing, an operation result of the xe2x80x9chorizontal processing modexe2x80x9d in which the input pixel data are processed in the horizontal direction, and an operation result of the xe2x80x9cvertical processing modexe2x80x9d in which the input pixel data are processed in the vertical direction.
This filter operation apparatus includes the register which contains the values for performing the xe2x80x9croundingxe2x80x9d according to the respective processing, and has a function of adding a value for performing the rounding with the addition means which can receive four values. Therefore, the degradation in accuracy is suppressed.
According to third embodiment, the filter operation apparatus of first embodiment comprises fourth pixel delay means for delaying the input pixel data for a predetermined period which is at least equal to or longer than the delay time of the first pixel delay means, and outputting the delayed data; fifth pixel delay means for delaying the output data of the fourth pixel delay means for a period which is as long as the delay time of the fourth pixel delay means, and outputting the delayed data; third selection means for selectively outputting one of the output data of the first pixel delay means and the output data of the fourth pixel delay means, in place of the output data of the first pixel delay means, to the first multiplication means and the second multiplication means, in accordance with the mode switch signal; and fourth selection means for selectively outputting one of the output data of the second pixel delay means and the output data of the fifth pixel delay means, in place of the output data of the second pixel delay means, to the first selection means.
This filter operation apparatus also can share the operation unit in the horizontal processing apparatus and the vertical processing apparatus even when the horizontal processing and vertical processing for the input image data is successively performed only in one direction. Therefore, the circuit scale of the hardware part can be reduced, whereby the scale of the filter operation apparatus can be reduced.